The object tracking algorithm based on discriminative classifier usually adopts complex appearance model to improve the tracking precision in complex scenes, which relatively influences the real-time performance of tracking. To solve this problem, a binary feature based on halftone was proposed to describe the object appearance and the kernel function of structured output Support Vector Machine (SVM) was improved, so as to realize fast updating and discriminating of discriminative model. In addition, a discriminative model updating strategy based on part matching was proposed, which can ensure the reliability of the training samples. In the experiments conducted on Benchmark, compared with the three algorithms including Compressive Tracking (CT), Tracking Detection Learning (TLD) and Structured Output Tracking with Kernels (Struck), the proposed algorithm had better performance in tracking speed with the increases of 0.2 times, 4.6 times and 5.7 times respectively. On the aspect of tracking precision, when overlap rate threshold was set to 0.6, the success rate of the proposed algorithm reached 0.62, which was higher than the success rates of the other three algorithms that were all less than 0.4;when the position error threshold was set to 10, the precision of the proposed algorithm reached 0.72,while the precisions of the other three algorithms were all less than 0.5. The experimental results show that the proposed algorithm obtains good robustness and real-time performance in complex scenes, such as illumination change, scale change, full occlusion and abrupt motion.